Hardware accelerators provide an opportunity to achieve orders-of-magnitude performance and power improvements with customized circuit designs. As technology has advanced, however, so has the amount of data needing to be processed, stored and transmitted. So-called big data is a big part of today's technology solutions, and with it, advanced solutions for compression and decompression so that more data can be stored and transmitted in less space (or requiring less bandwidth) than ever before. The challenge exists, however, of detecting silent data corruption in hardware engines that perform compression. The issue of soft errors (SER) is known, but these are generally detectable. The problem is severe, however, when an error goes undetected during a compression operation. Algorithms that generate highly compressed streams suffer from the problem that a corrupted stream is very hard to recover data from; in the worst case, all data after the point of corruption is lost.
Most current solutions rely on simply hardening the structures used in the compressors, such as error correction code (ECC)-protected RAMs or parity protected buses. But, if there is an undetected multiple-bit error, or an event in the computation data-path logic, it is not clear whether these can be avoided except using probabilistic methods, which are by their nature inexact. Some developers claim to have developed a full decompression operation as an error check on compression, but this solution is expensive as it requires significant hardware resources and also adds significant latency to an application's processing pipeline.